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ON VISCOSITY SOLUTION OF HJB EQUATIONS WITH STATE 
CONSTRAINTS AND REFLECTION CONTROL 


ANUP BISWAS, HITOSHIISHII, SUBHAMAY SAHA, AND LIN WANG 

Abstract. Motivated by a control problem of a certain queueing network we consider a 
control problem where the dynamics is constrained in the nonnegative orthant Kj. of the 
d-dimensional Euclidean space and controlled by the reflections at the faces/boundaries. 
We define a discounted value function associated to this problem and show that the value 
function is a viscosity solution to a certain HJB equation in R+ with nonlinear Neumann 
type boundary condition. Under certain conditions, we also characterize this value function 
as the unique solution to this HJB equation. 
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1. Introduction 

The chief goal of this article is to characterize the value function for discounted control 
problem associated to certain constrained dynamics (see (12.IIP as a viscosity solution of 
certain Hamilton-Jacobi-Bellman (HJB) equation with nonlinear Neumann type boundary 
condition. The controlled diffusion takes values in Ml, non-negative orthant of the Euclidean 
space M. d . The control process is matrix valued and represents the reflection vectors at 
certain time. This problem is motived by queueing network problem where there are d 

Date: November 29, 2016. 

2000 Mathematics Subject Classification. 93E20, 60H30, 35J25, 49L25. 

Key words and phrases. Skorokliod map with reflection control, queues with help, viscosity solutions, 
nonlinear Neumann boundary, non-smooth domain, heavy-traffic, stochastic network. 

1 








2 


ANUP BISWAS, HITOSHI ISHII, SUBHAMAY SAHA, AND LIN WANG 


classes of customers in the queueing system with d number of servers, for every server there 
is a priority class of customers, and customers from a non-priority class may access the 
server provided its priority class is empty at that instant (see Section l2Jl for more details). 
One of the interesting features of this model is its control process which acts through 
reflection. To the authors knowledge such model has not been studied before. For existence 
and uniqueness results for such dynamics we follow M- The constrained dynamics in m 
does not have any control component and in fact, the author assumes certain regularity 
properties on the reflection matrix in the time variable. We show in Theorem IA. II that such 
regularity is not needed to establish existence of a unique adapted solution. 

The value function is defined to be the infimum of a certain discounted cost function 
(see (12.41) 1 where the infimum is taken over all admissible controls. Admissible controls are 
processes that do not predict the future. Discounted cost functions are quite common in 
queueing theory (see [T] and references therein). The discounted cost considered here has 
two components (see ([2.31) 1. the running cost £ € C po i(R(|) and the cost hi € C(M. d (g) M. d ), 
with i := {1,..., d}, associated to the reflections . We show that the value function is 
in Cp 0 i(M^) and is a viscosity solution to the following HJB equation (see (12.51) 1 


CV - pV + £ = 0 in 
Ui(DV(x)) = 0 for x € dM+, i € I(x), 


( 1 . 1 ) 


where 

'Hiip) := min {p • Me, + hi(M)} , iGl, 

Me M v 

M is a certain subset of M. d (g) R rf , and I(x) denotes the collection of indices i € X for 
which Xi = 0. We refer [6J for more details on viscosity solutions. We note that the 
boundary condition here is of nonlinear Neumann type. The major hurdles in obtaining 
the uniqueness of viscosity solution are the non-smooth property of the boundary and 
the nonlinear nature of the boundary condition. The authors of EJ[DE] study viscosity 
solution in a non-smooth domain but with linear Neumann type boundary condition. In [5j 
viscosity solution framework is used to characterize the value function of an ergodic control 
problem with dynamics given by controlled constrained diffusions in a polyhedral domain. 
Nonlinear Neumann boundary condition is studied in mm (see also the references therein) 
for domains with regular boundary (at least C 1 ). The key idea in proving uniqueness of 
viscosity solution is to construct a suitable test function. We could establish the uniqueness 
under some assumptions on hi (see Theorem 12. li b)). The construction of test function also 
uses the structure of nonlinear boundary condition. 

The organization of this article is as follows. Section [2] introduces the control problem 
settings and the main results. Then in Section [3] we establish regularity properties of the 
value function and prove that the value function is a viscosity solution to (II.ip . while the 
uniqueness of a viscosity solution to (11.11) is established in Section [4] under the assumption 
of the existence of a right test function. Section [5] is devoted to establishing the existence 
of a test function. Finally, some auxiliary results are proved in the Appendix. 

Notations: By W l we denote the d-dimensional Euclidean space equipped with the 
Euclidean norm |-|. By {ei,..., e^} we denote the standard orthonormal basis in R d . We 
write X for the set {1,... , d} and set R^ = {x = (xi,..., x n ) € R^ : Xj > 0 for all i € X}. 
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We denote 1 = (1,..., 1) € R+. We denote by R d ®R d the set of all dxd real matrices and we 
endow this space with the usual metric. For a, b G R we denote the maximum (minimum) of 
a and b as aV6 (aA&, respectively). We define a + = aVO and a~ = —a AO. Trace of a matrix 
A € R d <S> R rf is denoted by Tr A. For x € R+ we define B r (x) = {y € R+ : \y — x\ < r} for 
r > 0. Given a topological space X and B C X, the interior, complement and boundary of B 
in X is denoted by B°, B c and dB, respectively. By B(X) we denote the Borel a-field of X. 
Let C([0, oo) : X) be the set of all continuous functions from [0, oo) to X. By Cq([0, oo) : R+) 
we denote the set of all continuous paths that are non-decreasing (component-wise) with 
initial value 0. We define C 2 (R rf ) as the set of all real valued twice continuously differentiable 
functions on W l . C po i(R+) denotes the set of all real valued continuous functions / with at 
most polynomial growth i.e., 


lim sup 
\ x \—>0 





0 for some k > 0. 


Infimum over empty set is regarded as +oo. C, K\, K 2 , ■ ■ ■, are deterministic positive con¬ 
stants whose values might change from line to line. 


2. Setting and main result 


We consider the problem in the domain R+. Let (II, J 7 , P) be a complete probability 
space on which a filtration o satisfying the usual hypothesis is given. Let (Wt, J^t) 

be a d-dimensional standard Wiener process (Brownian motion) on the above probability 
space, i.e., the Wiener process Wt is adapted to Bt, t> 0, and for every t,s > 0, Wt+ S — Wt 
is independent of J-). Let b : -A R d be a mapping and X be a d x d matrix satisfying 

the following condition. 

Condition 2.1. There exists a K € (0,oo) such that 

| b(x) — b(y) | < K\x — y\ and |6(x)| < K for all x, y € R+. 

Also the matrix A = T,T, t is non-singular. 

We note that Condition EU in particular, implies that X is non-singular. Let a € [0, l] d . 
We define 

M(a) := {M = [I — P] € R d <S) R rf : Pa = 0, P t] > 0 and^~~] Pji < a*}, 

i¥=i 

where I denotes the dxd identity matrix. We endow that set M(a) with the metric of 
(g) R^ and therefore M(a) forms a compact metric space. We assume that 


Condition 2.2. 


max on < 1. 


In what follows, a is assumed to be fixed and now onwards we write M(ct) as M. Given 
x € R^ we consider the following reflected stochastic differential equation 

r t r t 


f 


X(t)=x+ f b(X(s)) ds + T>W(t) + f v(s) dY (s), t > 0, 
Jo Jo 


( 2 . 1 ) 


Xi(s) dYi(s) = 0 for iel, X > 0, Y > 0, t > 0 , 
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where the process v takes values in the compact metric space M, and v(s,u) is jointly 
measurable in (s,w). We also assume that the control process v is {Ft} adapted. It 
is easy to see that v is non-anticipative : For s < t, W(t) — W{s) is independent of 
a{W{r),v(r) : r < s} as W(t) — W(s) is independent of F s . We call the control v 
an admissible control and the set of all admissible controls is denoted by 9JT. We show in 
Theorem lA.ll in Appendix that if Conditions 12.11 and 12 . 21 hold then for every v £ 9JI, (12.ip has 
a unique adapted strong solution (X,Y) taking values in C([0, oo) : R+) x Cq([0,oo) : R+). 

It is well known that for every non-decreasing continuous function 4> : R+ —>• R + , we can 
define a er-finite measure d(j) on ([0, oo), £>([0, oo))) where dcj)((a, 6]) = cj)(b) — <f>(a) for 0 < a < 
b < oo. For any / € L 1 ([0, oo), £>([0, oo)), dcf)) we denote Jj 0f j f(s)dc/)(s ) = f(s)dcj)(s). Let 
t be a non-negative continuous function with polynomial growth, i.e., there exists m, q € 
[0, oo) such that 

0 < l{x) < ci (1 + \x \ m ), x £ R+. (2.2) 

Let hi : M —>• R, i £ X, be continuous. Since M is compact, hi,i £ X , are bounded. Note 
that hi can as well be negative. Then for any initial data x £ Rl|_ and control v £ VJl, our 
cost function is given by 


J(x, v ) 


:= E x 



e /3s (£(X(s)) ds + ^2hi(v(s)) dYi(s)) , 

i£l 


(2.3) 


where /3 > 0, and (X,Y) is the unique adapted solution to (12.11) . We define the value 
function as 

V(x) := inf J(x,v). (2.4) 

v&XH 

It is shown in Lemma [3.1 1 below that V(x) is finite for all x £ R(|_. The goal of this article is 
to characterize the value function V as a viscosity solution of a suitable Hamilton-Jacobi- 
Bellman (HJB) equation. To define the HJB equation let us first introduce the following 
notations, 


C := i Tr(AD 2 ) + &•£>, 


where A = EE T and D and D 2 are the gradient and Hessian operators, respectively. Define 
for p £ R rf , 

'Hiip) := min {p ■ Met + hi(M)} , i£l. 

Me M 


For x £ R(^_ we define I(x) := {i £ X : X{ = 0}. Therefore I(x) denotes the indices of 
faces Ft := {x £ R(^_ : x* = 0} where x lies. Note that I(x) can also be empty. Indeed, for 
x £ R+, 


= 0 if x € (R+) ° , 

^ / 0 if x £ <9R^. 
The HJB equation we are interested in is given as follows 


I(x) 


CV-fiV + £ = 0 in (r+)°, 

Hi(DV(x)) = 0 for x € <9R+, i £ J(x), 


(2.5) 


The solution of (12.51) is defined in the viscosity sense ([6H8]) as follows: 
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Definition 2.1 (Viscosity solution). Let V : —>• M be a continuous function. Then V 

is said to be a sub (super) solution of (12.51) . whenever <p € C 2 (M rf ) and V — p has a local 
maximum (minimum) at x € R+, the following hold: 


(. Cip(x ) — (3V(x) + £(x)) V max 'Hi(Dip(x)) > 0, 

iei(x) 


x ) — j3V (x) + i(x)) A min 'Hi(Dip(x)) < 0, 

iei(x) 


respectively J. 


We will impose the following condition on hi. 

Condition 2.3. For any p € R rf and x € if we have maXj gI ( x ) TLi(p) < rj for some 
r) € (—oo, 0) then 


n iei (x){ M e M : p ■ Met + hi(M ) < g/2} / 0. 

Remark 2.1. Condition 12.31 is assumed purely for technical reasons. This condition will be 
used to show that V in (12.41) is a subsolution to (|2.5D . It is easy to see that Condition 12.31 
is satisfied if hi(M) = hi (Mu ,..., M^). 

Our main result characterizes the value function V given by (12.4)1 as the viscosity solution 
of (12.51) . Recall that C po i(M+) denotes the set of all continuous functions, defined on 
with polynomial growth. Our main result is the following. 


Theorem 2.1. Let Conditions [KTl\2.2\ and \2.3\ hold. Then the following holds. 

(a) The value function V given by (|2.4|) is a viscosity solution to the HJB given by (12.51) : 

(b) Assume that hi(M ) = c • Me* for some constant vector c = (ci,..., Cd). Then V is 
the unique viscosity solution to (|2.5|) in the class o/C po i(R+). 

Proof of Theorem I2.1f al is done in Lemma 13.11 Lemma 13.21 and Theorem 13.11 These 
results are proved in Section [3l Let us mention again that the uniqueness of viscosity 
solution does not follow from existing literature. We construct an appropriate test function, 
as stated in Theorem 14.21 in Section (5) which leads us to the uniqueness result. 


2.1. Motivation from queueing network. The motivation of the above mentioned prob¬ 
lem comes from a control problem in multi-class queueing network. Consider a queueing 
system with d customer classes where the arrivals of the customers are given by d inde¬ 
pendent Poisson processes {E 1 ,... ,E d }. There are d servers and every customer class has 
service priority in one of the servers. Therefore we can label the servers from 1 to d so that 
customer of class i has priority for service in server i for i € X. A customer of class j, j i, 
may access the server i if there are no customers of class i present in the system. We assume 
that the network is working under a preemptive scheduling, i.e., service of a non-priority 
class customer is preempted by a priority class customer. Such queueing systems might be 
referred to as queueing networks with help. Similar type queueing network is studied in [1] 
for a M/M/N+M queueing network with ergodic type costs. The service time distributions 
of the customers at different servers are assumed to be exponential. If we denote by X\, 
the number of class i customers in the system at time t, then we have the following balance 
equation 


x; = x* + e‘ - s‘(n 


f 


Bids)- Y,S ij (h ij 
j-jji 


Bi j dli), i € 1. 


( 2 . 6 ) 
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In (12.61) . {S l , S lJ ,i, j e 1} are independent standard Poisson process, /T- 7 (//*) denotes the 
service rate of the class i (class i) customers at the server j,j / i [i, respectively). B l s 3 ( B l s ) 
is an adapted process which denotes the status of service of class i customers at server j 
( i, respectively) at time s. H*- 7 = 0 implies that class i customer is not receiving service 
from server j whereas l?*- 7 e (0,1] implies that the class i customer is being served at server 
j with fraction of effort B*- 7 . Therefore we have 

0 < £ B\ j < 1. 


Due to priority it is easy to see that B l e {0,1}. /* denote the cumulative time when there 
were no customers of class i present in the system, i.e., 

n ■■= i ei. 

Jo 

Therefore I 1 is an increasing process and integrations in (|2.6I) make sense. We assume that 
every class is in heavy traffic in the sense that the arrival rate of class i customers is equal 

ji 

to //. Then we would have —>• 0 as n —>■ oo, and t > 0. We define the diffusion scaling 

as follows. 

E l t ■= ~^=(E l t — n l nt), S l (t) := -^={S\nt) - nt), S %3 (f) := ~^=(S 13 (nt) - nt ), 

V n V n V n 


XI ■= 


ypn 






n 


P 

1 nti 


P — —P 

1 t 1 nf 

n 


Hence using (|2.6D and a simple calculation we obtain 


XI 



(2.7) 


for i el. We also have 

XI dP s = 0 for alii el, t> 0. 

Now formally letting n —>• oo, in (12.7|) we have: {X, I) —> [Z, Z) and 



Zi = 4 + - w*’ 2 + jiZ 1 



zi dZl = 0 , 


Vi > 0, 



Vij(s)dZ 3 , 


( 2 . 8 ) 


where {W*’ 1 , IT*’ 2 , * > 1} are collection of independent Brownian motion with suitable 
variance. The relation between m and (12.81) is quite formal, in fact, the convergence 
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might not hold for any choice of control Bn . However, this relation can be justified for a 
reasonable class of controls. Now if we redefine Z l as /TZ 1 then from (12.81) we get 


Zl = Zi + W i + Z i -Y' [ — 


Vij(s)dZ J s , 


L 


ZldZl = 0, 


Vi > 0, 


(2.9) 


for some Brownian motion (W 1 ,..., W d ) with suitable covariance matrix and v takes values 
in M(l). Assume that —p < 1 for all i.j. Then it is easy to see that (12.91) is a particular 
case of m and Condition 12.21 is also satisfied. 

We can associate the following cost structure to the above queueing system. 


V(X 0 ) 


inf E 



-Ps 


(i(x s )ds+Y,cidii)\, 

i£l 


for some non-negative constants Cj, i G X, where the infimum is taken over all adapted 
process B. We note that if we let n —>■ oo formally then the above cost structure is a 
particular form of ()2.4|) . Cost structure of above type has been used to find optimal control 
for various queueing networks (see for example [I] and references therein). The importance 
of such control problems comes from well studied Generalized Processor Sharing (GPS) 
networks mm- In a single server network working under GPS scheduling in heavy-traffic, 
the server spends a positive fraction of effort (corresponding to their traffic intensities) 
to serve each customer class when all classes have backlog of work, otherwise the service 
capacity is split among the existing classes in a certain proportion. It is easy to see that 
our model considers a full-fledged optimal control version of the GPS models. 


Remark 2.2. The above relation between the queuing control problem and the value function 
defined in (12.41) is bit formal. We neither study the convergence of the value functions 
corresponding to the queuing model, nor try to find any asymptotic optimality result. These 
are topics of further study. The main goal of this article is to characterize the value function 
(12.41) in terms of the solution of HJB (12.51) . However, in the many server settings above 
questions are addressed in [3] . The limiting HJB in [3] has a classical solution which is used 
to obtain an optimal control for the limiting problem. This optimal control is then used 
to obtain convergence result of the value functions for the queuing model and asymptotic 
optimality. It should be noted that the solution of (12.51) is obtained in viscosity sense which 
gives additional difficulty in establishing convergence results of value functions. 


3. Proof of Theorem 12. If a) 

Conditions ED and m will be in full effect in the remaining part of the article. We start 
by proving the growth estimate of V given by (12.4j) . 

Lemma 3.1. Let V be given by (12.41) . Then there exist c\ G (0, oo) such that 
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nt d rt d 

i(t) = Xi+ / bj(X(s)) ds + S ijWj(t) + / y ~Yjj(s) rfY)(s), f > 0. (3.1) 

• /n !=i ' 70 l=i 


Proof. Fix x € R+ and v € VJl. Let (X,Y) be the solution of (12.11) given the control v and 
initial data x. Then we have 

d pt d 

xdt) = . ‘ . 

1=1 

Therefore summing over i we have from (13.11) that 

pt d d 

Y,x t (t) = j2 x i+ E^( x ( s )) o!s + E(E^) t lw + E[/ E%( s ) dY ?(s) 

iex «ex iex j =l V iex 1=1 L ^° *ex 

(3.2) 

Since v takes values in M we have Y!i v ij — 1> thus using Condition 12.11 and (13.21) we get a 
constant > 0 such that 

y' xdt) < ^ ] Xj+/C2 1 + y~] (i Wj(t)i+ Yi(t)) 
i£i *ex iex 

Now use (1A.3P to obtain 

(3.3) 


^2 xdt) < E x * + fC3 ( t+ E sup 


0<s<£ 
zEX — — 


«GX zGX 

for some constant ac 3 . Recall that for any a* > 0, * = 1,..., d + 2, and m € [0, 00 ) we have 

d +2 d+2 

(E“ i ) m£ <‘ i+2 > <m_1)+ S»r- 


2=1 


2=1 


Since A > 0 we have from (12.21) and (J3.3D that 
^(X(t))<Q + Q Vd\x\ + n 3 (t + sup |Wj(s)|) 


■ 0<s<t 

2 £X 


< c £ + c £ (d + 2 )( m - 1 ) + d m/2 |xr + c £ (d + 2 )( m - 1 ) + K 3 ft m + E sup |Wi(s)r) 

V ^ 0<s<t > 

<Q(d + 2)( m " 1 ) + d m / 2 (l + |xr) + c £ (d + 2)( m - 1 ) + K 3 ft m + Y sup |Wi( fl )pV 


. _, 7 - 0 <s<t 
zEX — — 


(3.4) 


Now we can find constants Ki,i = 4, 5, 6 , depending only on m, so that 


E, 


/>00 /* 00 

/ e -^(t m + Y sup |Wi(a)| m )dt] <K 4 fl+ / e“^E[sup |VFi(s)| m ])ds 
Jo 0 <s<t - 1 k Jo 0 <s<l ' 

< K 5 (l + J™ e-PH^ds 

< « 6 , 

where in the second inequality we use Burkholder-Davis-Gundy inequality EH- Therefore 
combining the above display with (13.41) we obtain 


E 


r /*°° 1 

/ e~^ t l{X{t))dt < ^ 7(1 + |x| m ), 

'-Jo - 1 


(3.5) 
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for some positive constant kj. Let |/ij|oo := sup MeM \hi(M)\ and |6|i = su PxeR d \h(x)\. 
Then for each i € {1,..., d}, 


E, 


e~P t h i (v(t))dY i (t ) < |hi 

= | hi 
< Kg 


Ex 

I hi | 


1 — max; a. 


< K 9, 


poo 

/ e~P t dY i (t) 

Jo 

- POO 

/ /3e-^Yi(t)dt 

-Jo 

POO 

/ f3e~P t (\b\it + Y"' sup \Wi(s)\)dt 

Jo 0<s<t 

(3.6) 


for some constants Kg, Kg, where the second equality is obtained by a use of a integration- 
by-parts formula (this is justified by (IA.3I) and Law of Iterated Logarithm which confirms 
that maximum of a standard Brownian motion has polynomial growth almost surely), in the 
third inequality we use (|A.3jl . and last inequality can be obtained following similar estimate 
as above (display above (1 3.5 [ 1) . We also note that the constants above do not depend on i 
and v € 9JL Therefore we complete the proof combining (13.51) and (|3.6I) . □ 


Next lemma establishes continuity of the value function V. 

Lemma 3.2. The value function V given by (El is continuous in Ml. In particular, 
V G C P oi(M+). 

Proof. To show the continuity we prove the following: for any sequence {v n } € 9JT and 
x n —>• x € as n —>• oo, we have 

| J(x n , v n ) — J(x, v n )\ —> 0, as n —» oo, (3-7) 

where J is given by (12.31) . From the proof of Lemma 13.11 it is clear that 


lim sup sup Ex 

T—>oo v£9Jl 


poo _ 

/ e~P S (£(X(s))ds + Y, hi(v(s))dYj(s) 
, ' T iex 


= 0 . 


Therefore to prove (13.71) it is enough to show that for any T > 0, 


E, 


r T 

/ e- 0s (£(X n {s))ds + y2h i (v n (s))dY i n (s) 

Jo iex 

r T 

/ e-P s (£(X n (s))ds + V /i i (u n (s))dy i n (s) 

Jo 


0 , 


(3.8) 


as n —>• oo where ( X n , Y n ), (X n ,Y n ) are the solutions of (12.11) with the control v n and the 
initial data x n , x, respectively. It should be noted that both the solutions ( X n , Y n ), ( X n , Y n ) 
are governed by the control v n for all n. Also from (|3.4j) and the calculations in (13.61) we 
see that dominated convergence theorem can be applied to establish (13.81) if we can show 
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0 a.s., 

0 a.s. 


(3.9) 


the following hold, 

i r T r T 

/ e~P s (l{X n {s))ds- / e- ps (l(X n {s))ds 
' Jo Jo 

( T e~l“Y I hi(v n {s))iYns) - f '£e-f”h l (v"(s))dY?(s) 

iex iei 

We recall the maps (T. S ) from Theorem IA.11 We have 
Y?(t) = Ti(X n ,Y n )(t)= sup max{0 ,-Ui(X n ,Y n )(s)}, 

0 <s<t 

X?(t) = S i (X n ,Y n )(t) = Ui(X n , Y n )(t) + Ti(X n , Y n )(t), 

where U t {X n , Y n )(t) = (x n )i + [ bi(X n (s))ds + (XW)i(t) + V [ v%(s)dY?(s). 

Jo ^ Jo 

The above relation also holds if we replace ( X n ,Y n ),x n by (X n ,Y n ) and x, respectively. 
Let [-] s t denote the bounded variation norm on the interval [s,t]. Then using the result 
from j!6l pp. 171] we have for all i € X, 

V? - Y?\*t < |^ n («) - Y?(s )| + | Ui(X n ,Y n )(s) - Ui(X n , ¥”)(*)| 

+ {U i (X n ,Y n )-U i (X n ,Y n )] st . 

Now one can follow similar calculation as in (|A.4D to obtain 

[Y n - Y n ] st < ^|^ n (s) - ^(s)! + sKd\\X n - X n \\ s + Kd(t - s) sup || X n (r) - X n {r) || 

i&X re M 

+ max OLi [Y n - Y n ] st , 


which gives 

\Y n -Y n \st < 1 (j2\Y?(s)-Y?(s)\+Kds\\X' 

-L lXlcLX^ ' _ 


iei 


-X n \\ s +Kd{t-s ) sup || X n (r)-X n (r 

re[s,t] 

(3.10) 


Similarly, we get 

sup ||X n (r) - X n {r )|| < ||A n (s) - A n (s)|| + Kd(t - s ) sup ||X n (r) - X n {r )|| 

rE[s,t] rE[s,£] 

+(1 + maxa^W* - Y n ] st + ^"(a) “ *J"(s)| 

iex 

and using (13.101) we have 

sup || X n (r) - X n (r)\\ < (l + 2V(2 Affe) )(|| X n (s) - X n (s)|| + “ *?(s)|) 

re[s,t] 1 — maxj on 7 

+ Kd{t - s){ 1 +---) sup ||X n (r) - A n (r)||. (3.11) 

1 max* ol{ re[s,t] 

Now we choose 9 > 0 so that Kd6( 1 + . 2 ) < 1. Since x n —>• x, we obtain from (13.101) 

and (13.111) that ||A n — X n \\o, \Y n — Y n \g tends to 0 as n —>• oo. Similar convergence can 
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be extended on interval [0 ,kd],k is a positive integer, by an iterative procedure and using 
(|3.10p and (13.111) . This proves ()3.9p . □ 

Now we prove the dynamic programming principle for V. Recall that B r (x) denotes the 
ball of radius r around x in M^_. Let ( X , Y ) be a solution of (12.1|) with initial data x. Denote 

oy := infjt > 0 : X(t) ^ B r (x)}. (3.12) 

That is, <j r denotes the exit time of X from the ball B r (x). 


Proposition 3.1 (Dynamic programming principle). Let a = a r be as above. Then for any 
t > 0 we have 

rcrAt 

V(x) = inf E x ' 
v ; vem 


rcr/\t , 

/ e~ 0s (£(X{s))ds + y2hi(v(s))dYi{s)) + e~ 0aM V(X{cr ht)) . (3.13) 

"'° iei 


Proof. Using the Markov property of Brownian motion one gets that for any v € SO1, 


E, 


/‘OO -. 

/ e-^ s (£(X(s))ds + J2 h i(v)dYi) >e-^ M V(X(aAt)). (3.14) 

L i£l 


See for example m Lemma 3.2]. For s > 0, we have v € 911 such that 
V(x) > J(x,v) — e 


= E 


raAt 

/ e-P s (l(X(s))ds + J2h i (v)dY i )+ 
Jo inx 

/‘OO 

/ e-P s (l(X{s))ds + J2 h i(v)dYi) 
JaAt ieI 

a At 


> E x 




a At 


— £ 


per AT. 

/ e~ Pa (e(X(s))ds + V hi(v)dYi) + e~ 0aM V{X{a A t)) 
Jo iex 


- e, 


where in the last inequality we use (13.141) . Since e is arbitrary, we get 


V(x) > inf E* 

v ' vem 


roAt 


e~ ps {i(X{s))ds + s >"h i {v)dY i ) + e-P'™V{X((jf\t)) . (3.15) 

LJ0 J 

To prove the other direction we use Lemma IA.1I from Appendix which says that for given 
£ > 0, v € 911, there exist control v € SOI such that 


E, 


r OO 

/ e-^ s {e(X{s))ds + J2 h i^) dY i) X a At <e~^ M V{X(aAt))+3£, 

JaAt ieI 


and 


v(s) = v(s) for all s < a At. 


V{x) < E x 



( i{X(s))ds + ^ hi(v)dYi)+ 
iex 


Then 
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< E, 


i 


e~ Ps (£(X(s))ds + hi(v)dYi) + e~P aM V {X (a A t)) 


i£l 


+ 3e. 


Now we take the infimum over v to get the other side inequality. Hence (13.131) follows using 
(13.151) and the above display. □ 


Lemma 3.3. Given r, e G (0,1) there exists t G (0,1) such that 

sup P x (a r < t) < £ for all x € M+, 
vewi 

where ay is given by (13.121) . 

Proof. Fix a; € and v € 9H. Let ( X , Y) be the solution of (12.11) given the control v and 
initial data x. Then for iGlwe have 

Xi(t) = Xi + f b l {X(s))ds + (Y,W(t)) i + V f v^dYjis), t> 0. (3.16) 

Jo j=1 Jo 

Then summing over i we get from (13.161) that 


Xi(t) - Xi\ < t'Y sup \bi(x)\ + ^|(EW(t))»| + (1 + ma.xai)Y Y j(t) 

iex iex xeR + iex 1 iei 

< m (t sup \b(x)\ + V'sup|Wj(s)|'), 

v iei s ^ ' 


(3.17) 


for some constant n\, depending on E, a, where in the last inequality we use (IA.3I) . Now if 
| X(t) — x\ > r and t k\ sup xgIR d \b(x)\ < r /2 then using (13.171) we obtain 


r/2 < m y'sup|W i (s)|. 
iex 


Thus we have for the above choice t that 

supP x fsup|X(s) - x\ > r) < P(V sup|Wj(s)| > ) 

vetm y s<t J s<t 2fiq/ 

<hhi E [ s „p|»' 1 ( S )H 

X s<t 

16 gPk? 

< - 


where in the last line we use Doob’s martingale inequality. Thus we can further restrict t, 
depending on e, so that the rhs of above display is smaller that e. This completes the proof 
observing that P x (a r <t)< P x (sup s<t |X(s) — x\>r). □ 


Now we are ready to show that the value function V, given by (12. ip . is a viscosity solution. 
This clearly concludes the proof of Theorem 12.11 (a). 


Theorem 3.1. Assume that Conditions 1 2. llFOl hold. The value function V is a viscosity 
solution of the HJB given by (J2.5D . 
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Proof. First we show that V is a super-solution. Let <p G C 2 (M+) be such that V — ip attends 
its local minimum at x G We can also assume that V(x) = <p(x), otherwise we have to 
translate <p. Therefore we need to show that 

(. Cip(x ) — /3ip(x) + £(x)) A min 'Hi(Dip(x)) < 0. (3.18) 

xei(®) 

Let us assume that (13.18jl does not hold and we have a 8 > 0 such that 
(£tp(x) — /3ip(x) + £(x)) A min Hi(Dip(x)) > 28. 

xGl(x) 

Using the continuity property we can find r > 0 such that the following holds 
Cip(y) — /3<p(y) + £(y) > 9 for all y G B r (x), V — (p > 0 on B r (x), 

7£i(Dip(y)) >8 for y G B r (x), and I (y) C I (a;) for y G B r (x). 

From (13.191) we get 

Dip(y) ■ Mei + h{(M) > 8 for all M G M, z € I(y), y G B r (x). 

Let v G 591 and (X, Y") be the corresponding solution to (12.11) with initial data x. Dehne 
a = a r = inf{t >0 : X(t) £ B r (x)}. Apply Ito formula (TOj to obtain 

V(x) = ip(x) = E x [e^ aAt y>(X(a A t))] 

rcrAt 


(3.19) 

(3.20) 


- E T 


rcr At 

/ e~^ s (C(p(X(s))ds — /3ip(X(s))ds + Dip(s) ■ v(s)dY (s)) 

Jo 


< E x [e^ aM V(X(a At))} 


+ 


r rcrAt 

/ e _/3s (£(X(s))ds — 8ds — Dip(s) ■ v(s)dY (s)) 

L Jo 


where in the last line we use ()3.19D . Again using (13.201) we see that 


rcrAt 

Jo ' 


rcrAt 


e P s Dlp(X(s)) ■ v(s)dY(s) = / e l3s Dip(X(s)) ■ v(s)eidYi(s) 

iex Jo 

rcrAt 

> / e _/3s (6»- hi(v(s)))dYi(s), 


iez ' 


where we use the fact that / Q CT Si^i(X(s)) J^i( s ) = 0. Thus combining above two displays 
we obtain that for any v G 911, 

r rcrAt 

V(x) <E X e~^ At V(X(a At)) + / e~ Ps (£(X(s))ds + V hi(v(s))dYi(s)) 

L J o 


iez 


-E, 


rcrAt 

/ e“^(0ds + 0 VdYi(s)) 

■ /o zex 


Since v G SHI is arbitrary we have from above that 

r /-crAZ _ 

U(x) < inf E* e~ fiaAt V(X(a At)) + / e -^(£(X(s))ds + V hi(v(s))dYi(s)) -a(t), 

Jo cei 

( 3 . 21 ) 


reim 
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where 


ait) = inf E^ 

ve<m 


raAt 

/ e- ps (6ds + 0y"dY i (sj) 
Jo 


In view Lemma 13.31 we can find t so that a{t) > 0 which will give us from (13.211) 


Vix) < inf E x 

v ’ v&7l 


0 —/3 a At 


raAt 

V{X{aAt)) + / e~ fi9 (£(X(s))ds + 'Th i {v(s))dY i (s)) 

iei 


but this is contradicting to Proposition 13.11 This establishes (I3.18p . 

Now we show that V is also a subsolution. Let tp € C 2 (M. d ) be such that V — tp attends 
a local maximum at the point x € R+. Also assume that V(x) = p(x). We have to show 
that 

(Ctp(x) — /3ip(x) + t{x)) V max T~Li(Dip(x)) > 0. (3.22) 

ie i(x) 

If not, then we can find 9 > 0 such that 

(C(p(x) — P<p(x) + t{x)) V max / Hi(Dp>{x)) < —26. (3.23) 

i£l(x) 

By Condition 12.31 there exists M E M such that 

ma x{D<p(x) ■ Mei + hi(M)} < —9. 

iei(x) 


Therefore using (13.231) we can find r > 0 such that V(y) — ip(y) < 0 for y € B r (x) and the 
following hold: 

£<p(y) - P<p(y) + £{y) < - 0/2 for y e B r (x), 

{Dip(y) ■ Mei + hi(M)} < -9/2 for i € I (y), I (y) C I(x), and y € B r (x). 


Now we take the control v = M and follow the same calculations as above, and using (13.241) . 
to arrive at 


r raAt 

V(x)>E x e~^ At V(X{a A t)) + / e~ Ps (£{X{s))ds + V hi{v(s))dYi(s)) 
1 Jo tel 


+ Ba- 


raAt 

/ e~ ps {9/2ds + 9/2 J^dY^s)) 
Jo iex 


Now using Lemma [3.31 we know that the last term on rhs of above display is strictly positive 
for a suitably chosen t and therefore we obtain 


r rcrAt 

V(x)>E x e-P aAt V(X{cT At))+ / e~ Ps (t{X(s))ds + V hi(v(s))dYi(s)) 

iex 


but this is contradicting to Proposition 13.11 This proves (13.22[) . Hence V is a viscosity 
solution to (12.51) . □ 
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4. Proof of Theorem 12.Ilf b) 

In this section we show the uniqueness property of the viscosity solutions of (12.511 . Indeed, 
we prove the following theorem. 

Theorem 4.1. Let Conditions \2.1\ and \2.2\ hold. Assume that there exists a vector c = 
(ci,..., Cd) E such that hi(M ) = c - Me{ for all i El. If u E C po i(R^.) and v E C po i(R+) 
are, respectively, a viscosity subsolution and a viscosity supersolution to (|2.5D . then u < v 
on M+. 

Theorem 12.11 (b) follows immediately from the theorem above. 

Proof of Theorem \2.1\ (b). According to Theorem l2.ll (a) (or Theorem 13. ip and Lemma 13.11 
assure that the value function V given by ()2.4jl is a viscosity solution to () 2 .5 jl in the class 
C P oi(R+)- Theorem 14.11 then guarantees that V is the unique viscosity solution to (12.511 in 
Cpoi(Mt). ' □ 


To establish the comparison theorem above, it is crucial to find a test function having a 
few of properties, stated in the next theorem. 

Theorem 4.2. Assume that hjfM ) = 0 for all i E X. Then there exists a globally C 1,1 
function ip on having the properties: 

V(y) > 0 for all y EW l \ {0}, 

if(Xy) = A 2 tp(y) for all y E M. d and A > 0, 


and for each i EX, 


Hi{Dtp(x)) 


> 0 
< 0 


if Xi > 0, 
if Xi < 0. 


We note that if d > 1 and hi = 0 for some i EX, then 

TiAp) = min p ■ Mei = pi — a* max pf, 
MeM(a) jEZ\{i} J 

while if d = 1 and h\ = 0 , then 

Hi ip) =P- 

The proof of the theorem above is given in the next section. 


Proof of Theorem \4-l\ Let u E C po i(M^_) and v E C po i(M+) be, respectively, a viscosity 


subsolution and a viscosity supersolution to 

First of all, we note that it can be assumed that c = 0 and (5 = 1. Indeed, setting 
u±(x) = u(x) + c • x and v\{x) = v(x) + c • x, observing that w := u\ (resp., w := vi) is a 
viscosity subsolution (resp., supersolution) to the HJB 


Cw — fhv + l{x) + c • (—b(x) + fix) = 0 in (M+)°, 

min Dw ■ Met = 0 for x E cMl, i E I(x). 
MgM(a) 
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This boundary value problem can be rewritten as 

f 0~ l Cw — w + /3 _1 (£(x) — c ■ b(x)) + c ■ x = 0 in (M+)°, 

1 min Dw ■ Met = 0 for x £ cMl, i £ I(x). 

^MgM(a) 

Thus, replacing u, v, A, b and £ by u\, v\, 0~ 1 b and 0~ 1 (£ — c • b) + c- x, if necessary, 

we may assume that c = 0 and 0 = 1. 

Henceforth we assume that c = 0 and 0=1. In particular, we have hi = 0 for all i £ X. 
Since u,v £ C po i(M+), there are constants m £ N and k > 0 such that 

|u(x)| V |v(x)| < k(1 + \x\) m for all x £ M+. 

Let e > 0 and set for x £ M+, 

= (m + l )- 1 ^ 1 + • • • + x™ +1 ) + 1 • x, 
u £ (x) = u{x) — e{C + ip(x)) and v e (x) = v(x) + e(C + tp(x)), 

where C is a constant to be fixed soon. 

Note that ip € C 2 (M+), 

lim u e (x) = —oo and lim v e (x) = oo, (4-1) 

| cc | —>-oo |x|—>oo 

and observe that for some constants C % > 0, with i = 1,2, 3, and for all x £ M+, 

W(x)\ < C0x\ m , \Cip(x)\ < C 2 (l + \x\) m and \Cip(x)\ < C 3 + ip(x), 

where we have used the fact that b is bounded on M(j_. 

Now, we fix C = C 3 . Let x £ and (p,X) £ J 2,+ u £ (x). See ( 6 ] for the definition of 
semi-jets J 2,± . Setting ( q,Y ) := (p,X) + £(Dip(x), D 2 tp(x)) £ J 2,+ u(x), we compute that 

Tr AX + b(x) ■ p — u £ (x) + £{x) 

> Tr AY + b{x) ■ q — u(x) + £{x) — e\Cip{x)\ + e{C + ip(x)) (4.2) 

> Tr AY + b(x) ■ q — u(x) + £{x). 


Recalling that 


Hiip) = pi — oti max pf for all i £ X, 
j¥=i J 


and noting that for any i £ I, 

Diip(x) = x™ — 1 for all x £ M+, 
where Di = d/dxi, we see that if Xi = 0 , then 

'Hiip) = qi+ £ - oti max (qj + e(—x^ 1 + 1)) + 

> qi + e(l - cti) - ai max q+ = 7ii{q) + e(l - a*). 

j^i ■> 

We thus deduce that w := u £ is a viscosity subsolution to 

j Cw — w + £ = 0 in (R+)°, 

\t-L~(D w(x)) =0 if x € 9M+, i £ I(x), 


(4.3) 
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where 'H i (p) := T~Li{p) - e(l — «;)• 

Similarly, we see that w := f| is a viscosity supersolution to 

( Cw — w + £ = 0 in (R+)°, 
\%f(Dw(x)) = 0 if 2 : 6 9R|, i G I(s), 


(4.4) 


where Uf (jp) := %(p) + e(l - a*). 

It is enough to show that for any e > 0, we have u £ < v £ on R+. 

To do this, we fix e > 0 and suppose to the contrary that sup (u £ — v £ ) > 0. Let 
p G C 1 , 1 (M rf ) be the test function given by Theorem 14.21 For any k G N, consider the 
function 

4>(x, y) = u £ (x) - v £ (y) - ktp(x - y) on (x, y) G ^R+) , 

and let (xk,yk) be a maximum point of this function 4>. Note that the existence of such a 
maximum point is ensured by (ED. 

Since (p > 0 is globally C 1,1 on M rf , we find that for all (x,y) G (R+) and some constant 
C > 0, 

<p(x - y) < <p(x k - y k ) + Dip(x k - y k ) ■ (x - y - (x k - y k )) + C\x - y - (x k - y k )| 2 . 


It is standard to see (see [ 6 J for instance) that 


lirn kip(x k - y k ) = 0 

k—> 00 


and that there are symmetric matrices X k , Y k € R d such that 




< 6 Ck 



(4.5) 


(4.6) 


and 

(Pk,X k ) € J 2 ’ + u £ (x k ) and (p k , Y k ) G J 2 ~v £ (y k ), (4.7) 

where p k = kD<p(x k — y k ) and J 2,± denotes the “closure” (see [ 6 ] ) of the semi-jets J 2,± . 
The inequality above implies that X k < Y k . 

Let x k = (x kll ..., x kd ) and y k = (y kl ,..., y kd ). If x ki = 0 for some 1 G X, then x ki -y ki < 
0 and hence 


'Hi C Pk ) = kHi(Dtp(x k - y k )) - e(l - a*) < -e(l - a*) < 0. 
Thus, by the viscosity property of u £ , we see that 

Tr AX k + b(x k ) ■ p k - u £ (x k ) + £(x k ) > 0. 


Similarly, we obtain 

Tr AY k + b{y k ) ■ p k - v £ {y k ) + £(y k ) < 0. 

Combining these two, we get 

u £ (x k ) - v e (y k ) < Tr A{X k - Y k ) + (b(x k ) - b{y k )) ■ p k + £{x k ) - l{y k ) 
< K\x k - y k \\p k \ +£(x k ) - £(y k ), 


(4.8) 
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where K is a Lipschitz bound of the function b. Furthermore, observe that {x k , y k } C (®+) 2 
is a bounded sequence by (14.111 and in view of the homogeneity of ip that for all z € and 
for some constant C 4 > 0, 

\z\\Dip(z)\ < C±ip{z), 


and, consequently, 

\xk ~yk\\Pk\< Cikip(x k - y k ). 

Thus, from (14.811 and (14.511 . we infer that 

0 < sup (u e - v £ ) < u £ (x k ) - v e {y k ) - ktp(x k - lIk) 


< K\x k - y k \\p k \ +i(x k ) -£(y k ) ->• 0 as k -a- 00 , 
which is a contradiction, and conclude that u £ < v £ on R+. 


□ 


Remark 4.1. The above proof also works if we replace the operator CV—j3V+£ by a general 
nonlinear operator satisfying some standard condition. One may look at [6l Conditions 
(0.1)-(0.3)], [3 Condition (1.3)], [8l Condition (1.4)] for general form of such operators. 


5. Construction of a test function 


We now give the proof of Theorem 14.21 That is, we construct a test function in the class 
C 1,1 (M d ), which is a main step in establishing the comparison principle. 

When d = 1, the function ip(x) = x 2 obviously has the properties of test function stated 
in Theorem 14.21 We may thus assume in this section that d > 1. 


5.1. A convex body. We set 

E d := {£ = (£i,... ,60 : & = 1 or cu for all i € 1}. 

For £ = (£l, ..., £ d ) € E d , we set 

£ # = #{i€Z:& = 1}, 

where #A indicates the cardinality of the set A, and, for k € {0,1,... , d}, 


We set 


and note 


=t ■■= {{ 6 2" : (* = U- 
=i := = \ {(“!■• 


fcex 


Let 

and assume that 


#Zt = 2 d -l. 

. ru 1 


1 = 0i <... < e d , 

d — 1 + maxjgj Oj 


?! > 


Note that using Condition 12.21 we have 


d 


d- 1 + maxjgj a* d max !eI o-j _ 

0 1 > - 3 - 0 d > ---= max a, fc/ d . 

a a iex 


(5.1) 

(5.2) 
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For ^ £ 5^, we define 

H(0 -.= {xeR d :Z-x<d k }, 

and, for 5 > 0 , set 

HJ := {x G M n : Xi > —5 for all 1 < i < d}, 

s ■= n h ®> 

and 

5,5 :=SCHg. 

Note that 

&H(£) = {x G R d : £ • x = 9 k } for £ G E d k . 

Lemma 5.1. The set Ss is a compact convex subset of R d and is a neighborhood of the 
origin. 

Proof. It is clear that 5^ is a closed convex subset of R d . Observe that 

$5 C HJ 0 H(l), 

and Hf 0 H( 1) is bounded in R d . Hence, Ss is a compact subset of R d . 

The distance from the origin to the hyperplane 

dH(0 = {x G : £ • x = 9 k }, 


with £ G E d , is given by 


M£l- 


Hence, setting 

P := & A min{0fe/|£| : £ G E% , k G 1} (< S), 

we find that 

B p { 0) C S s . 

Thus, Ss is a neighborhood of the origin. 

Let z G dSs and let N(z, Ss) denote the normal cone of Ss at z, that is, 
N(z, Ss) := {p G R d : p ■ (x — z) < 0 for all x G 5^}. 


□ 

(5.3) 


Set 

and 


J z .— {? G I . Zi — d}, 


:= {£ G Et : z G dtf (£)}■ 
According to [T5J Corollary 23.8.1], we have 

Lemma 5.2. We have 

N(z,Ss) = conical hull (E z U {—ej : j G J z }) = 

j&Jz 


l+(-ej) + R + coE z , 


where coS. denotes the convex hull of E z . 
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For the time being we examine the normal cone N(z,S ) at z for z € dS n SR^. Note 
that 

3R+ = [J F t , 

i£l 

where Fi :={i£ R^ : Xi = 0 }, and therefore 


N(z,S) = R+ coE z . 


Lemma 5.3. Assume that z € F 1 ,. n dS. For any £ € coS 2; we have 

£i = a.i and max£,- = 1. 

jGX 

Proof. For notational convenience, we treat only the case when i = d. Let £ € coE z . In 
order to show that £^ = a^, we suppose to the contrary that £^ / otd- This implies that 
there exists £ G E z such that £d = 1. We select k E X so that £ E E d . Set 

£ = (£i, • • • , id-i, otd). 

First we consider the case when k > 1. Note that £ E Ef _j and that 

£ • z < and £ ■ z = 9^. 


The last two relations are contradicting, since < 0*, and 

£-z = £- (zi,...,z n _i, 0 ) = £ ■ (zi,... ,z n _i, 0 ) = £ ■ z. 
Next consider the case when k = 1. Note that 


£ = («!,... ,«rf) 


and 


#i = £- z = £- z< 


z < max a,-' 
“ iex 


z < max a* 9d, 
i£l 


which contradicts Q>- 
Next we show that 


max£j = 1. 
i£l 

We argue by contradiction, and suppose that 


max£j < 1. 
iex 

Since £ E co E z , there exist £ 1 ,..., 6 H 2 and A j > 0, with 


N 




where we may assume that N > d. 
Writing 

e = (&■• 


N 

such that £ = Aj£*, 

i— 1 


,£d) for i E iV}, 


max F < 1, 
i£l 


by the supposition that 
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we find that for each i G Z, there exists ji G {1,..., N} such that 

£f = «i- 

We can rearrange (£ 1 , Ai),..., (f N , Xn) in such a way that N > d and ji < d for all i£l. 
For iel, let fcj £ I be such that £* € . Observe that for any i£l, 


and hence 


e-z = e ki , 


;*£?•* = sIX as 


iex 


d 


iex 


Since z£5nFiCF(l)n Z^, we have 


d^ 


z < 


(d — 1 ) + max cii (d — 1 ) + max a. 


iex 


d 


1 • z < 


d 


■6L. 


Thus combining above two displays, we get 

(d — 1 ) + max«j 


h < 


d 


e d . 


This contradicts m- 


□ 


Lemma 5.4. The multi-valued map z H > H. is upper semicontinuous on dS. That is, for 
each z G dS there exists e > 0 such that 


Proof. Take z G dS and put 
Since 


E x C E z for all x G B e (z) 0 dS. 

“ Z := “* \ “2- 

z 0 dH(f) for any £ € S'; 


and jfzX < oo, there is e > 0 such that 


That is, we have 
and, consequently, 


dist (z,dH(£)) > e for all £ G S°. 
B e (z) n 9id(£) = 0 for all £ € E°, 
H x C S, for all a; G B s (z) 0 dS. 


□ 


Lemma 5.5. There exists 5 > 0 such that for each i G I, 

£i = ctj and max£, = 1 for all £ £ coS 2 and z € 95 0 F t g, 
jel 

where F iy g = {i £ R d : Xj > —6 for all jGl, |xj| < 5}. 





22 


ANUP BISWAS, HITOSHI ISHII, SUBHAMAY SAHA, AND LIN WANG 


Proof. Let z G dS n F^. By Lemma 15.41 there exists e := e z > 0 such that 

r. x C H z for all x G B e (z ) H dS. 

This yields 

co^ C coS z for all x G B e (z) n dS. 

The family {B e (z)} is an open covering of the compact set dS n Fj. Hence, there exists 
5d > 0 such that for any x G dS n Fg t g d , there is 2 € dS n Fg such that 

CO^i Cl CO 

which implies that 

fd = and max£j = 1 for all f G coHa;. 

i£l 

Similarly, we may choose 6i > 0 for each i€l such that for any x € dS H F^g. , 

F = a.i and max£,,• = 1 for all f S coS x . 

j&x 

Thus, setting 5 := minwe conclude that for any i G 1 and x G dS H F^g, 
fi = a.i and maxL = 1 for all f G coH x . 

□ 


5.2. Sign of PLi. Let us recall that for iGl, 


PLiip) = Pi ~oti ma xpf. 


jpi ‘ 3 


Set 


Bi } g = {x G : \xi\ < (5} for i Gl. 

Lemma 5.6. Let 5 > 0 be the constant from Lemma 15.51 Let i G 1 and x G dSg n Ei t g. 
Then 

B-iil) < 0 for all 7 G N(x, Sg), 
where N(x,Sg) is given by (15.51) . 

Proof. We set J := {j G T : Xj = —5}. Note that J or may be empty set. We note that 

N(x,Sg) = 22 ~^-+ e j + M+ co Ha,. 
jeJ 

Let 7 G N(x,Sg). We choose A j > 0, A > 0, and f = (£ 1 ,..., fg) G coHj, so that 


By Lemma 15.51 we have 


7 = 22 —Aj e i + A£. 
jeJ 

fj = atj for all j G J, 


fi = OL{ and maxf 7 - = 1. 

jex 


and 
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In particular, we have Cfc = 1 for some k ^ J U {i}. Thus, we have 

7 = - Aj)e_j + Ae fc + ^ A&ej 

jeJ /ex\(Ju{fc}) 


Now, if i € J, then 

Hi (7) = Aaj - A, - a*A = - A, < 0, 

and, if z 0 J, then 

Hi(7) = A«i - «jA = 0. 

□ 

Set 

Gi = {x € R d : Xi > —<5}. 


Lemma 5.7. Let <5 > 0 be as before. Let i € X and x € < 95,5 hi G*. ITien 

( 7 ) > 0 for all 'y€N(x,Ss). 

Proof. Set J := {j € X : Xj = —5}. Note that J may be an empty set and that, since 
x € Gi, we have i fL J. Let 7 € N(x,Ss), and recall that 

N(x, S$) = y^M + (-ej) + R + co~ x . 
j&J 

Choose Aj > 0, A > 0, and ^ G coH x so that 



7 = X] ~ X 3 e i + 

ieJ 

and observe that 

7 = /* — A i) e i T ^ ^ A^efc 

j&J k£l\J 

and that 

otk < £k < 1 for all k € X. 

Hence, 



Hi (7) = 7i - «i max 7 + > AC; - a, maxA£ fc > Acii - a* A max > 0. 

k^i k^i k^i 

□ 

Remark 5.1. Combining Lemmas 15.61 and 15.71 we see that for any x € dSs, if \xi\ <5 for 
some i, then 


Hj( 7 ) = 0 for all 7 eN(x,Sg). 
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5.3. Construction of tp(x). For 0 < e < <5, let 

<Sf := jx € : dist (x, S$) < e| , 

where 5 is the constant from Lemma 15.51 It is clear that <S| satisfies the uniform interior 
sphere condition. Thus we have the following lemma. 

Lemma 5.8. The boundary <95| is locally represented as the graph of a C 1,1 function. 

Proof. Fix any z € <9<S|. We show that cAS| can be represented as the graph of a C 1,1 
function in a neighborhood of z. Making an orthogonal change of variables, we assume that 
2 : = \z\e d = (0,... ,0, \z\). 

Let B' r (x') denote the open ball in of radius R > 0 and center at x' € M d_1 

and let B' R := B R ( 0). We choose r > 0 so that B 3r x {0} C (<Sf)° Recall that 5| is a 
convex, compact, neighborhood of the origin of M d , note that, if x' € B 3r , then the set 
{x d : (x',x d ) € 5f} is a closed, finite interval containing the origin of R, and set 

g(x') = max{x d : (x',x d ) € <Sf}. 

Note that 0 < g(x') < M for all x' € B 3r and some constant M > 0. 

We show that g € C 1,1 (R'). Since <Sf is convex, g is concave on B' 3r . This together with 
the boundedness of g implies that g is Lipschitz continuous on B' 2r . Let L > 0 be a Lipschitz 
bound of g on B' 2r . 

Fix any x' € B' r . By the definition of <S| and the convexity of S 5 , there exists a unique 
y € Ss such that 

B e (x',g(x')) n S 5 = {y}- 
On the other hand, since y € Ss, we have 

B e {y) c Sf. (5.4) 

Now, using a simple geometry and the Lipschitz property of g, we deduce that 

\x^ — I 

g(x') > yd and J r -r- < L. 

9 {x') - y d 

Indeed, the second estimate above can be obtained as follows: due to the continuity of 
x' eA (x',g(x')) eA y, it is enough to establish the above relation at point x' where V g 
exists. Convexity of Ss and <Sf implies that (x 1 — y',g(x') — y d ) € N((x', g(x')), <S|) and 
N((x',g(x')),SI) = {A (—X7g(x'), 1): A € K + }. Thus for some A > 0 we have (a/ — y',g(x') — 
yd) = Vi?(x'), 1). Now it is easy to see that the second estimate above holds. Further¬ 

more, we get 

e 2 = |z' - yf + (gix 1 ) - y d f > (1 + L~ 2 )\x' - yf. 

Thus, setting 6 := Lf\Jl + L 2 € (0, 1), we have 

\x - y | < Oe, (5.5) 

Note that V := B' r r\B' e {y') is a neighborhood of x'. Let z' € V and observe by (15.411 that 

\(z',g(z'))-y\>e = \(x’,g(x'))-y\. 

From this, noting that g(z') > y d , we get 

g(z') > g{x) + \Je 1 — \z' — y '\ 2 — \Je 1 — \x' — y '\ 2 


(5.6) 
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Next setting h(z') = \J e 2 — \z' — y '\ 2 for z' € V and using (15. 5 j) . we calculate 

1 


D 2 h{x') = - 


s 2 — \x' — y'\ 2 


^ + O' - y') ® O' - y') 


\Je 2 — \x' — y'\ 2 

where I' denotes the (d — 1) x (d — 1) identity matrix, and moreover, 


D 2 h{x') > — 


1 + 


£0 2 


I'. 


e 2 (i - e 2 ) V 

This and (15.611 show that g is semi-convex on B' r , which together with the concavity of g 
ensures that g £ C l,1 {B' r ). 

Let p [0, +oo) be the gauge function of «S|, defined by 

p{x) := inf{A > 0 : x € A5f}, (5-7) 


□ 


where 

\S$ := {A£ : £ € Sf}. 

It is well known that /?(•) is a positively homogeneous, of degree one, convex function. In 
particular, we have p(0) = 0. It is clear to see that p(y) > 0 at y / 0. 


Lemma 5.9. The function p is locally C 1,1 on R d \ {0} and for y £ R d \ {0}, Dp(y) € 
N(y/p(y), S$), where N(y/p(y),S %) denotes the normal cone at y/p(y) € (AS|. 

Proof. It follows from the definition of p that for any !/ £ K' ! \ {0} and A > 0, we have 

A = p(y) if and only if y/X £ dSf. (5-8) 

Fix any 2 £ R d \ {0} and show that p is a C 1,1 function near z. Up to an orthogonal 
change of variables, one can assume that 2 = \z\ed- It follows from Lemma 15.81 that there 
exists a neighborhood V of the origin in M fi_1 and a function g £ C 1,1 (U) such that for any 
x' € V and x d > 0, 

(x',xd) € dSf if and only if Xd = g(x'). (5-9) 

Set A 2 = p(z). By (|5.8|> and (15.911 . we have 

Zd/K = g(z '/\ z ) = g( 0). 

Set I = (A z /2, 2A 2 ) and W = (j^)V x (0, oo), and note that W x / is a neighborhood of 
(z, A 2 ) € R^ +1 . For any y = (y', yd) € W, in view of ([5.811 . we intend to hnd A € I such that 

y/ As ^iSf. 

Since y 7 /A 6 V, the above inclusion reads 

Vd/X = g(y'/X). 


We dehne the function F : W x I —>• R by 

/(y,A) = y d /X- g(y'/X), 
note that f(z, X z ) = 0, and calculate that 

df, , . z d 
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By the implicit function theorem, there are a neighborhood U C W of z and a C 1,1 function 
A : U I such that f(y,A(y)) = 0 for all y G t/. In view of (15.81) and (15.91) . we see that 
p(y) = A(y) for all y G U, which completes the proof. 

Now, let y E M d \ {0} and p = Dp(y). Note that y/p{y) E 5<Sf and p = Dp(y/p(y)) and 
that for any x E <Sf, 

1 > p(x) > p(y/p{y)) +p-(x- y/p{y)) = 1 + p ■ (x - y/p{y)), 
that is, we have 

p ■ (x — y/p(y)) < 0 for all x E <Sf. 

This shows that 

Dp(y) =p G N(y/p(y),S$). □ 


Lemma 5.10. Let x € dSg and id. For 

f < 0 

\>0 


any 7 € N(x,S £ 5 ), 
if Xi < 5 - e, 
if Xi > -(6 - e). 


Proof. Fix any x G dS$. 

Since p{x) = 1, p(z) < 1 for all z G 5|, and p is convex, we have 

1 > p(z) > p(x) + Dp{x) ■ (z — x) for all z G <Sf, 

which implies that Dp(x) G N(x,S$). Since p is differentiable at x and Dp(x) / 0, it is 
easily seen that 

N(x,Sg) =M + Dp(x). (5.10) 

Set y := x — eDp{x)/\Dp(x)\ and observe that 

B £ (x) n dS s = {y}, (5.11) 

which implies that B e (x) 15 5^ = {y}, and therefore 

Dp(x) G N(y,S s ). (5.12) 

Indeed, it is clear that 

B e {x) n dSs / 0. 

Note moreover that if z G B e (x) D dS$, then B e (z) C <S| and, according to (15.101) . 

Dp(x) ■ (w — x) < 0 for all w G B e (z). 

This last inequality readily yields 

z — x = —eDp(x)/\Dp(x)\, 

that is, z = y, and we conclude that (| 5.11 [) holds. 

Fix any 7 G N(x,Sf) and iGl By (|5.10l) and ([5.12D . we have 

7 = (M/I Dp{x)\)Dp{x) G N(y,S s ). 


Thus, noting that 


Vi 


< Xi + \x — y\ < 6 
> Xi — |x — y\ > —5 


if Xi < 5 — e, 
if Xi > — (d — e), 
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we may apply Lemmas 15.61 and 15.71 to get 


Ki( 7) 



if Xi < 6 — e, 
if Xi > — (5 — e), 


which completes the proof. 


□ 


Proof of Theorem \4-2\ As noted before, we know that when d = 1, the function p(x) := x 2 
has all the required properties. 

In what follows we assume that d > 1, and we define 

p(x) := p 2 (x ) for x € R d , 

where p is the function given by (15.71) . It is clear that p(x ) > 0 for all x € \ {0} and p 

is positively homogeneous of degree two. 

Next we show that p is globally C 1,1 on M d . Since p is locally C 1,1 on M d \{0}, the function 
p is also locally C 1,1 on M d \{0}. Let x, y € R^\{0} and in view of the homogeneity property 
of p we obtain that 

Dp(x) - Dp(y) = \x\Dp(x/\x\) - \y\Dp{y/\y\) 

= \x\(Dp(x/\x\) - Dp(y/\y\)) + (|s| - \y\)Dp(y/\y\). 

We choose a constant C > 0 so that 


\Dp(jp)\ < C and \Dp(p) — Dp(q)\ < C\p — q\ for all p,q € dB\. 

Combining these yields 

I Dp{x) - Dp(y )| < \x\ \Dp(x/\x\) - Dp(y/\y\)\ + |(|®| - \y\)Dp{y/\y\)\ 

< C\x\ \x/\x\ - y/\y\\ + C\x - y\ 

< 2C\x - y\ + C\x -y\ = 2,C\x - y\. 

This shows that for any x,y € 

I Dp(x) - Dp(y )| < 3C\x - y\. 

Accordingly, the function p is globally C 1,1 on W l . 

Let x € \ {0} and i£l, and assume that Xi > 0. By Lemma 15.91 we have 

Dp(x) = 2 p(x)Dp(x) € N(x/p(x),S e 5 ). 

Noting that x/p(x) € <95|, we see from Lemma 15.101 that 

'H i (Dp(x)) > 0. 

Thus, noting that Dp( 0) = 0 and, hence, TLi{Dp{ 0) = 0 for all i € X, we may conclude that 
for any x € W l and i € T, 

/ Hi(Dp(x)) >0 if Xi > 0. 

An argument similar to the above shows that for any x € M d and i € X, 

/ Hi(Dp(x)) <0 if < 0. 


This completes the proof. 


□ 
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Remark 5.2. The C 1,1 regularity of test functions is sufficient to achieve the comparison 
theorem as is shown in the proof of Theorem 14.11 Alternatively, one can build smooth test 
functions by using the mollification techniques. 

Appendix A. 

Let us first recall the Skorokhod problem which will be used in the analysis below. 

Definition A.l (Skorokhod Problem). Given 0 £ C([0,oo) : M) with 0(0) > 0, a pair 
(0,?0 € (C([0,oo) : M +)) 2 is said to solve the Skorokhod Problem (SP) for 0 on the domain 
[ 0 , oo), if 

( 1 ) <!>(*) =ip{t)+v(t), t>o, 

(2) 0(0) = 0(0), t > 0, 

(3) r] is non-negative, non-decreasing and f Q cj>(s)dr](s) = 0, V t > 0. 

In fact, there is always a unique pair (0, rj) solving the Skorokhod Problem for a given 
il’, given by 

r](t) = sup (— 0 (s)) + , 0 (f) = 0 (f) + r/(t), t > 0 . 

se[o,t] 

Recall that for a = (aq,..., a^) £ M+ with max; Oj < 1, we have 

M(a) = {M = [I — P] £ R d <g> R d : Pa = 0, Pij > 0 and ^ Pji < a,}. 

In this section we show that for any v € and given x £ Mi, the following reflected 
diffusion has a unique adapted strong solution (X,Y) £ C([0, oo) : M^.) x Cg([0,oo) : M^_), 

X{t) = x + J b(X(s))ds + T,W(t) + [ v(s)dY(s), t > 0 , a.s., (A.l) 


[0,i] 


/ 

J [ 0 ,oo) 


Xi{s)dYi(s) = 0 for all i € X, 


where VP is a standard d-dimensional Brownian motion on a given filtered probability space 
(fl, J 7 , {Jq}, P). Here fUt denotes the set of all admissible controls v where v takes values in 
M(a). Before we state the result let us define the following quantity, 

Pi(t) = sup max{0, (—SVP(s))j}. (A.2) 

o <s<t 

The following result is a modified version of the results obtained in 1111 - 

Theorem A.l. For any v £ 911, (IA.1I1 has a unique adapted strong solution (X,Y) taking 
values mC([ 0 , oo) : M^_) x Cg([0, oo) : M+) and 

1 




< 


i€l 


1 — max,; ai 


p +A 

iex 


for all t > 0 , a.s., 


(A.S) 


where (3 is given by (IA.2|) and | 6 |i = Xigx su PxeK d \bi( x )\- 


We refer, for instance, to El for general results for reflected diffusion processes on smooth 
domains. 
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Proof. The key ideas of this proof are borrowed from pH] . Fix T > 0. To show the 
existence of a unique adapted strong solution it enough to show the existence of a unique 
adapted strong solution on the the interval [0, T\. Thus we consider the space C([0, T] : 
R d ) x Cj([0,T] : M^). For (x,y) G C([0,T] : R^) x Cj([0,T] : R*), we define 

||x|| r = Yj sup |iCi(s)|, 
iei se [°> T l 

[u\t = 

iex 

where we recall that [-] s denotes the bounded variation norm on the interval [0, s]. For 
positive 71 , 72 , we endow the space C([0,T] : R+) x Cq([ 0, T] : R+) with the metric 

d ((x,y), (x,y)) = ji\\x - x|| T + 7 2 [y - y] T , 

where (x, y ), (x, y) G C([0, T] : R+) xCg([0, T] : R+). It is easy to check that for any positive 
7 i, i = 1, 2 , C([0, T] : R+) x Cq([0, T} : R+) forms a complete metric space with the metric d. 
For (x, y) G C([0,T] : R+) xCg([0, T] : R+) we define two maps T : C([0, T] : R+) xCq([ 0,T] : 
R+) -> Cj([0,T] : R d ) and 5 : C([0,T] : R^_) x C}([0,T] : R+) -> C([0,T] : R^) as follows: 

Ti(x,y)(t) = sup max{0,-I7j(x,y)(s)}, 

0<s<t 

Si{x,y)(t) = Ui(x,y)(t) +%(x,y)(t), 

where t/j(x, y)(t) = Xi + / bi(x{s))ds + (T,W)i(t) + V' / Vij(s)dyj(s). 

Jo • Jo 

3^ 

From the property of 1-dimensional Skorokhod problem we note that any fixed point of 
( S,T ) solves (|A.1I) . The idea of the proof is choose suitable 7 i,To so that the map ( S,T ) : 
C([0, ToJ : R+) x Cg([0, To] : R+) —> C([0, To] : R d ) x Cq([ 0, To] : R+) becomes a contraction 
almost surely. Once we have uniqueness on the interval [0, To] one can extend the result 
on [0, T] be standard time shifting argument. Now we fix a sample point in (fl,T,P). 
Then we view Xq,W as deterministic elements. Let K be the Lipschitz constant of b. Let 
(x,y),(x,y) € C([0, T] : R d ) x Cq([0,T] : R+). Using the lemma of Shashiasvili [TB] pp. 
170-175] concerning variational distance between maximal functions, we get 

[%(x, y) - %(x , y)] T < [Ui(x, y) - Ui(x, y)\ T 

f T ( T 

< \bi(x(s)) - bi(x(s))\ds + V / (~Vij(s))\dyj - dyj\(s), 

Jo ^ Jo 

where we have used the property that Vij < 0. We sum over i to obtain 


d , T 

[T(x, y) - T(x, y)\ T <KdT\\x-x\\ T + Y, / ^2(~ v ij( s ))\dyj ~ <%|(s) 

J=1 i+3 

d ,.t 

< KdT ||x — x|L + maxaj > / \dyj — dyA(s) 

1 U Jo 
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= KdT [lx — x|L + max a* [y — y\x- 

1 i 

Similarly, 

||«5(x,y) -5(x,y)|| oo < 2KdT\\x - x\\ t + 2 maxaj [y - y] t- 

Therefore combining (1A.4I) and (1A.5D we obtain 

d ((S(x,y),T(x,y)), {(S(x,y),T{x,y))) 

/ ^JTn( 2 7l+72) || _|| , (271 +72) f -j 

< A dT - 7 i|p - x\\ T + maxaj- -- 72 ly - y]|r- 


(A.4) 

(A.5) 


7i 


72 


( 271 + 72 ) ( 271 + 72 ).,,, , ,_ 

< maxjAfll-, maxaj-}d((x, y), (x, y)). 

71 i 72 

Now we first choose 71,72 to satisfy maxj aj < 1 (use the fact that maxj a* < 1) 

and the choose T small enough to satisfy KdT + 7l ^+ 72 ) < 1 . Hence we have 

d((S(x,y),T(x,y)), ((S{x,y),T{x,y))) < gd((x,y), (x,y)) for some g < 1. 

Therefore one can use contraction mapping theorem to get the unique fixed point in 
C([0, To] : K d ) x Cg([0, To] : W l ) that solves (1A.1I) . 

Now we prove (1A.3D . Let (X,Y) € C([0, 00 ) : M+) x Cq([0, 00 ) : M+) be the solution of 
(1A.1D . Then 


where 


Yi(t) = sup max{0, — Ui(X, Y)(s)} for all i € T, 

0 <s<t 


Ui(X,Y){t) = xi+ f b i (X s )ds + (XW) i + V [ v ij {s)dY j (s). 
JO , , . JO 


j¥=i 


Since x € Ml we have from (1A.2I) 


Ui(X,Y)(t) < t sup |6j(x)| + Pi(t) v ij(s)dYj(s ) 

iGR*? a-la J 0 




Thus using the fact Vij < 0 for i / j we obtain 


Yi{t) < t sup | 6 j(x)| +fii(t) - ^ / Vij(s)dYj(s ) 




i¥* 


Summing over i and using the property that u(s) € M(a) we have 
£*(t) - t l^l 1 + ^ Pi(t) + maxaj ^Tj(f), 

i€Z i i£I 


and this proves (IA.3I) . 


□ 


Recall J and V from (12.31) and (12.41) . Also recall a from Proposition 13.11 
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Lemma A.l. Let e > 0 and x € M. d . Then for every v € VJl, there exists a control v € !DT 
such that 


E.t 


o~P S ( 


' a At 


(£{X(s))ds + Y,h i [v)dY i ) <e~^ M V{X{a At)) + e 


iex 


a.s. 


Proof. Recall the ay is the exit time from the open ball B r (x). Now using the continuity 
property of V (Lemma 13.211 and (|3.7I) we can find d > 0 so that 

sup| J(y,v) - J(y,v)\ + \V(y) - V(y\ < e for all y,y € B r (a;), (A.6) 

vew. 

and \y—y\ < 5. Now consider a hnite Borel partition {D k }k>i of B r (x) such that diam(Dj) < 
5. Choose yk € Dk■ Then by definition we have v k € 9JT satisfying 

J(Vk,v k ) < V(y k ) + e. 


Combining with ()A.6D we see that for any y S D k , we have 

J{y,v k ) < J(yk,v k ) +£ < V{y k ) + 2 e <V(y) + 3e. (A.7) 

Since v k is adapted to {F t } we have a progressively measurable map Tj : [0, oo) xC([0, oo) : 
M rf ) —> M such that 

v k (t) = ^k(t, W{- A t)), a.s. Vt > 0. 

Now for any v € DJI and the corresponding solution (X,Y) define 

\ _ j v(s) if s < a A t, 

1 S \ Tfc (s — a A t, W{- A s) — W(cr A t)) if s > a At, and X(a At) € D k . 


By strong uniqueness we see that a At does not change with the new control v. Then 
denoting Y = Y(a A t + •) — Y(a A t), we obtain 


E, 


poo _ 

/ e-^^Xis^ds + Y^hiivWi) 
L ^ aM iex 




at\t 


= e~ f3aM E 


poo 

/ e^ 13s [£(X(t7 A t + s))ds + 'y' hi(v(a A t + s))dYf) 

Jo id 


X, 


a!\t 


= e ' J(X(a A t), v(a A t + •)) 


< e-^ At V{X{a At)) + 3e, 
where in the last inequality we use (1A.7I) . 


□ 
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